Skip to content

feat(desktop): add Pocket voice settings - #3208

Open
johnmatthewtennant wants to merge 13 commits into
jt/buzz-voice-refactorfrom
jtennant/pocket-tts-settings-v1
Open

feat(desktop): add Pocket voice settings#3208
johnmatthewtennant wants to merge 13 commits into
jt/buzz-voice-refactorfrom
jtennant/pocket-tts-settings-v1

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Context

Pocket TTS can read live agent replies during a huddle, but Buzz does not provide an installation-wide way to turn that behavior off or choose among the bundled voices. The setting also needs a stable voice identity boundary that future backends and per-agent preferences can reuse without treating a display label as identity.

Summary

This adds a compact Pocket TTS voice settings surface with an Agent text to speech toggle, preview playback, and 12 bundled voices. Text to speech remains enabled by default with Mary selected. Disabling it immediately cancels queued and in-flight agent speech.

Changes

  • Adds a versioned, device-local TTS settings schema and backend-qualified voice keys.
  • Bundles Anna, Azelma, Charles, Eponine, Eve, Fantine, George, Jane, Mary, Michael, Paul, and Vera with provenance and deterministic Mary fallback behavior.
  • Adds voice selection and preview controls while preserving the saved voice when text to speech is disabled.
  • Labels the visible and accessible selector as Pocket TTS voice.
  • Speaks only new live agent-authored text in active huddles, including kind 40002 agent replies, while excluding history, people, reactions, edits, system or tool rows, and attachment-only content.
  • Keeps agent speech strictly live-only across reconnects. The limit: 0 subscription cannot page missed history or reorder replay against new live events.
  • Removes the unused speakableAgentText wrapper and replaces stale replay-override coverage with a live-only no-paging assertion.
  • Cancels stale, queued, and in-flight speech on barge-in, shutdown, or voice changes without appending stale audio.
  • Adds content-free route diagnostics, focused Rust and frontend coverage, and a Playwright screenshot flow.
  • Groups huddle audio settings into a cohesive state boundary and extracts relay timing constants so both existing files remain below their strict file-size ratchets.

Related issue

Testing

  • Focused alias-aware frontend suites: 33 passed, 0 failed across TTS routing, voice logic, huddle filtering, and reconnect replay.
  • cd desktop && pnpm exec tsc --noEmit: passed.
  • Biome on all nine final-patch files: passed.
  • Strict file-size, px-text, and pubkey-truncation guards: passed.
  • git diff --check: passed.
  • Fresh cd desktop && pnpm build:e2e: passed with 4,556 modules transformed.
  • Fresh voice-settings.spec.ts --project=smoke: 2 passed, 0 failed.
  • Normal pre-push checks passed, including branch-skew, Desktop checks, and 3,784 Desktop tests.
  • The prior published native surface passed full Tauri tests and clippy. The final patch changes only TypeScript and E2E files.
  • Final affected Rust, frontend, Stacksmith, NCMB, and dead-code reviews found no actionable issues.
  • Exact validated head: ea833b1fb036e133f9be8ec64de7d34fae1bcd0c.
  • Exact validated base: 388cf0ccdce8e627be47dd335241987286ab9bf0.

Manual validation remaining:

  • In a live dev or release huddle, audition all 12 bundled voices.
  • Confirm a new kind 40002 agent response is spoken through the selected huddle output.

Deferred follow-ups:

  • Make Pocket model and voice installation explicit in the Voice card, including installed state, exact download size, progress, success or error state, and retry. Preview may start installation but must expose its progress.
  • Diagnose the occasionally silent first live huddle utterance with content-free signals for cold app or session state, route admission, native queue acceptance, worker readiness, player append or start or drain, and sink teardown. Keep this separate unless deterministic evidence identifies a regression in this change.

Screenshots

The expanded 12-voice selector screenshot is preserved locally at desktop/test-results/voice-settings/pocket-voices.png with SHA-256 a430ce71e7093612a0e9d27ebd3c6e43b2fa02b081c28376d8292e50f45d0a8b. It has not been posted.

Reviewer-reproducible examples

From a fresh checkout, run the supporting frontend contracts:

. ./bin/activate-hermit
cd desktop
pnpm test
pnpm check
pnpm exec tsc --noEmit
pnpm build:e2e

Observed result for the exact head:

Focused routing, voice, huddle, and reconnect suites: 33 passed, 0 failed
Desktop tests: 3784 passed, 0 failed
Desktop checks and TypeScript: passed
E2E build: 4556 modules transformed

Exercise the expanded selector and live-only subscription contract:

cd desktop
pnpm exec playwright test tests/e2e/voice-settings.spec.ts --project=smoke
node --import ./test-loader.mjs --experimental-strip-types --test src/features/huddle/lib/ttsLiveMessages.test.mjs src/shared/api/relayReconnectReplay.test.mjs src/shared/api/relayChannelFilters.test.mjs src/features/settings/ui/voiceSettingsLogic.test.mjs

Observed result:

Voice settings smoke: 2 passed, 0 failed
The agent speech subscription remains limit:0 and does not request missed-history paging
12 bundled voice options render under the Pocket TTS voice label

The remaining user-visible audio check requires a live huddle and physical output device:

  1. Open Settings, select each bundled voice, and use Preview.
  2. Join a huddle with an agent and select the intended output device.
  3. Send a message that produces a new kind 40002 agent response.
  4. Confirm the response is spoken once through the selected output and that changing the voice cancels stale audio.

@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-settings-v1 branch from df8cad5 to a741f1a Compare July 27, 2026 23:41
@johnmatthewtennant
johnmatthewtennant changed the base branch from main to jt/buzz-voice-refactor July 27, 2026 23:41
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-settings-v1 branch 2 times, most recently from 4891a92 to c118f5a Compare July 28, 2026 04:48
@johnmatthewtennant
johnmatthewtennant force-pushed the jt/buzz-voice-refactor branch 5 times, most recently from be4adc9 to 617e0d2 Compare July 28, 2026 21:40
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-settings-v1 branch from 7e5cc4b to 7cc0804 Compare July 28, 2026 21:48
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-settings-v1 branch from 7cc0804 to acb302e Compare July 29, 2026 00:24
johnmatthewtennant and others added 13 commits July 29, 2026 12:20
Signed-off-by: John Tennant <johnmatthewtennant@gmail.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/pocket-tts-settings-v1 branch from ea833b1 to 6186fc2 Compare July 29, 2026 16:37
@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review July 29, 2026 19:21
@johnmatthewtennant
johnmatthewtennant requested a review from a team as a code owner July 29, 2026 19:21
@johnmatthewtennant johnmatthewtennant added triage-ready Appropriate for agentic review and removed triage-ready Appropriate for agentic review labels Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant